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Abstract. After the Anshel-Anshel-Goldfeld (AAG) key-exchange protocol 
was introduced in 1999, it was implemented and studied with braid groups 
and with the Thompson group as its underlying platforms. The length-based 
attack, introduced by Hughes and Tannenbaum, has been used to extensively 
study AAG with the braid group as the underlying platform. Meanwhile, a 
new platform, using polycyclic groups, was proposed by Eick and Kahrobaei. 

In this paper, we show that with a high enough Hirsch length, the polycyclic 
group as an underlying platform for AAG is resistant to the length-based 
attack. In particular, polycyclic groups could provide a secure platform for 
any cryptosystem based on conjugacy search problem such as non-commutative 
Diffic-Hcllman, ElGamal and Cramer-Shoup key exchange protocols. 



1. Introduction 

The Anshel-Anshel-Goldfeld (AAG) key-exchange protocol was introduced in 
1999 pQ. Following its introduction, AAG was extensively studied using different 
groups as its underlying platform. Ko et al. used braid groups |12) , while Shpilrain 
and Ushakov [16] proposed Thompson's group. 

Hughes and Tannenbaum [9] originated the length-based attack (LBA) on AAG 
with its implementation in braid groups. They emphasized the importance of choos- 
ing the correct length function. Later, Garber et al. [6] gave several realizations of 
this approach, particularly a length function for the braid group and experimental 
results suggesting that the attack is infeasible given the parameters in existing pro- 
tocols. However, Garber et al. [5] also suggested an extension of the length-based 
attack which uses memory which succeeded in breaking AAG. Similar attack was 
implemented against a system based on the Thompson group [TS]. Most recently, 
Myasnikov and Ushakov [T3] analyzed reasons behind the failure of the previous 
implementations of the LBA, e.g. the occurrence of commutator-type peak, and 
gave an experimental evidence that the LBA can be modified to break AAG with a 
high rate of success. However, this work is again done with respect to braid groups. 

Meanwhile, a different platform for AAG, that of the polycyclic group, is sug- 
gested by Eick and Kahrobaei [5]. In polycyclic groups, the word problem can be 
solved efficiently, but known solutions to the conjugacy problem are far less efficient. 
Using experimental results, Eick and Kahrobaei showed that while the conjugacy 
problem can be solved within seconds using polycyclic groups with small Hirsch 
length, the conjugacy problem in polycyclic groups with high Hirsch length can not 
be solved even in a much longer time. 
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Taking inspiration from this result, we investigate the success rate of the length- 
based attack on polycyclic groups, especially those with high Hirsch length. Toward 
this end, we first construct polycyclic groups of high Hirsch length using a method 
introduced by Holt et al. |S]. Then, we implement the LB A using the algorithms 
presented in [5[[6j[14]. The experimental results that we collect suggest that with 
high enough Hirsch length, the polycyclic group as a platform for AAG is resistant 
to the known different variants of the LB A. 

As a wider application, we note that the conjugacy search problem is the basis 
for various cryptographic protocols besides AAG, such as the non-commutative 
Diffie-Hellman key exchange [12] , the non-commutative El-Gamal key exchange [ID] , 
the non-abelian Cramer-Shoup key exchange [2| and the non-commutative digital 
signatures [TTj . The LBA can be applied to all of these protocols; therefore, a 
secured platform group, proven resistant to known variants of the LBA, such as 
polycyclic groups, can help strengthen them. 

The paper is organized as follows. In Section^ we introduce the Anshcl-Anshel- 
Goldfcld key exchange protocol. In Section [31 we give a short review of polycyclic 
groups and the construction that we have used. In Section 31 we review the length- 
based attack, and in Section [SJ we present the experiments, results and conclusions 
that we have made. 

2. The Anshel-Anshel-Goldfeld Key Exchange Protocol 

Following [M], we introduce the Anshel-Anshel-Goldfeld key-exchange protocol. 
For more details, see pQ. As usual, we use two entities, called Alice and Bob, for 
presenting the two parties which plan to communicate over an insecure channel. 

Let G be a group with generators g±, . . . , g n . First, Alice chooses, as her public 
set, a — (ai, . . . , ajVi) where a, £ G and Bob chooses, as his public set, b = 
(&i, . . . , 6jv 2 ) where hi £ G. They both publish their sets. Alice then chooses 
her private key A = ■ ■ ■ a e s ^ where a Si 6 a and Ei G {±1}- Bob also chooses 
his private key B = b^ 1 ■•■bf^ where b ti € b and Si 6 {±1}- Alice computes 
b\ = A~ x biA for all 6; G b and sends it to Bob. Bob also computes a[ — B~ 1 a,iB 
for all ai Ga and sends it to Alice. Now, the shared secret key is K = A~ 1 B~ 1 AB. 
Alice can computes this key by 

K A = A- 1 a'^---a'^ = A-^B^a^Bf- ■ ■ ■ {B^a^B)^ 
= A- 1 B- 1 al 1 i ---a £ s L L B = A- 1 B- 1 AB = K 

Bob can likewise computes Kb = B~ 1 bf 1 1 ■ ■ -bf^ = B^ 1 A^ 1 BA, then the shared 
key is K = Kg 1 . 

In order to find K, the ear-dropper needs to find either A' 6 (a\, . . . , ajvj) such 
that V = A'- l bA' or find B' G {b u b N2 ) such that 'a 1 = B'^aB'. Thus, the 
security of AAG is based on the assumption that the subgroup-restricted simulta- 
neous conjugacy search problem is hard. 

3. Polycyclic Groups 

In this section, we give a short review of the polycyclic presentation and discuss 
how we generate polycyclic groups of high Hirsch length. For more details, see [8]. 
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3.1. The polycyclic presentation. Recall that G is a polycyclic group if it has 
a polycyclic series, i.e., a subnormal series G = G\ > G2 > ■ • • > G n +i = {1} with 
non-trivial cyclic factors. The polycyclic generating sequence of G is the n-tuple 
(<7i, . . . , g n ) such that Gj = Gj+i) for 1 < i < n. 

Every polycyclic group has a finite presentation of the form: 

(ffi, • • • ,5n I 9f = Wij, gf = Vij, g r k k = u kk for 1 < i < j < n and k 6 I) 

where Wij , , w^fc are words in the generators gi+i, . . . , g n and / is the set of indices 
i G {1, . . . , n} such that r.; = [d : Gj+i] is finite. Here a 6 stands for 6 _1 a6. 

Using induction, we sec that each element of G defined by this presentation can 
be uniquely written as g — g^ 1 ■ ■ ■ g^ 1 with e, £ Z for 1 < i < n, and < e, < Ti 
for i S I, This is the normal form of an element. If every element in the group 
can be uniquely presented in the normal form, then the polycyclic presentation 
is called consistent. Note that every polycyclic group has a consistent polycyclic 
presentation. 

The Hirsch length of a polycyclic group is the number of i such that = [Gj : 
Gj-i-i] is infinite. This number is invariant of the chosen polycyclic sequence. 

3.2. Polycyclic groups for A AG. Polycyclic groups are suitable as platform 
groups for AAG for several reasons. First, the word problem can be solved efficiently 
using the collection algorithm [3]. Second, the search conjugacy problem has no 
efficient solution in general polycyclic groups. This assessment is due to Eick and 
Kahrobaei [3J, using the following experiment: let K = Q[x]/(f w ) be an algebraic 
number field for a cyclotomic polynomial f w , where w is a primitive r-th root of 
unity, and let G(w) = O x U where O is the maximal order and U the unit group 
of K, r the order of w and h(G(w)) the Hirsch length. The average time used for 
100 applications of the collection algorithm on random words and the average time 
used for 100 applications of the conjugacy algorithm on random conjugates are: 



r 


h(G(w)) 


Collection 


Conjugation 


3 


2 


0.00 sec 


9.96 sec 


4 


2 


0.00 sec 


9.37 sec 


7 


6 


0.01 sec 


10.16 sec 


11 


14 


0.05 sec 


> 100 hours 



We can see that the collection algorithm works very fast even for polycyclic 
groups of high Hirsch length, which enables the word problem to be solved effi- 
ciently; but the solution to the conjugacy problem is not efficient for polycyclic 
groups having high Hirsch length. 

4. The length-based attack 

The length-based attack is a probabilistic attack against the conjugacy search 
problem in general, and against AAG in principal, with the goal of finding Alice's 
(or Bob's) private key. It is based on the idea that a conjugation of the correct 
element should decrease the length of the captured package. Using the notations 
of Section [2j the captured package is b' = (b[, . . . , b' N ) where b\ — A~ x biA. If 
we conjugate b' with elements from the group (ai, . . . , awi) and the length of the 
resulting tuple has been decreased, then we know that we have found a conjugating 
factor. The process of conjugation is then repeated with the decreased length tuple 
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until another conjugating factor is found. The process ends when the conjugated 
captured package is the same as b = (&i, . . . , &jv 2 ), which is known. Then, the 
conjugate can be recovered by reversing the sequence of conjugating factors. For 
more details on the length-based attack, see [T3l[T4] . 

4.1. Variants of the LB A. In [5l|6l[14l[T5] , several variants of the LBA are pre- 
sented. Here, we give four variants of LBA that we implemented against AAG 
based on the polycyclic group. In all these algorithms, the following input and 
output are expected: 

• Input: a, = (01, . . . , a^), b = (61, . . . , 6jv 2 ) and b' = (b[, . . . ,b' N2 ), such 
that b[ =bf for i = l,...,N 2 

• Output: An element A' G {ai, . . . , o^i) such that b\ — bf for i — 
1 , . . . , N2 or FAIL if the algorithm cannot find such A' 

The following notation is also used: if c = (ci, . . . , Ck), then its total length \c\ is 

4.1.1. LBA with backtracking. The most straight-forward variant of LBA (Algo- 
rithm [TJ) conjugates b' directly with af 1 € {ai, . . . , ajVi}- This is termed "LBA 
with backtracking" by Myasnikov and Ushakov |14j . 



Algorithm 1 LBA with backtracking 



1 

2 
3 
4 
5 
6 
7 
8: 
9 
10 
11 
12 



Initialize S = {(F, id G )}. 
while S ^ do 

Choose (c, x) S S such that |c| is minimal. Remove (c, x) 
for i = 1, . . . , Ni and e = ±1 do 
Compute 5i <e — |c| — \c ai \ 

if c Qi = b then output inverse of xa\ and stop 

if Si <e > then > length has been decreased 

Add (c a *,xa e t ) to S 
end if 
end for 
end while 

Otherwise, output FAIL > no more element to conjugate 



4.1.2. LBA with a dynamic set. Through analysis, Myasnikov and Ushakov con- 
cluded that different types of peaks make LBA unsuccessful [14 . To overcome this, 
they suggested a new version of the algorithm, which they termed "LBA with a 
dynamic set" . Here, depending on whether an causes a length reduction, either 
only the conjugates and products involving ai are added to the dynamic set, or, in 
the unlucky case of no length reduction, all conjugates and two generators products 
are added. Their experimental results suggest that this algorithm works especially 
well in the case of keys constructed from long generators, but not worse than the 
naive algorithm in other cases. Algorithm [5] is a modified version of their algorithm, 
which we implemented to attack AAG based on the polycyclic group. 
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Algorithm 2 LBA with a dynamic set 

1: Initialize S = {(VAcIg)}- 

2: while S ^ do 

3: Choose (c, x) S S such that |c| is minimal. Remove (c, a;) 
4: for j = 1, . . . , Ni and e — ±1 do 
5: Compute <Jj lE = |c| — \c Ui | 

6: end for 

7: if <5i. e < for all i then 

8: Define a cxt = a U {xiXjX^ 1 , XiXj, x\ \ Xi, Xj G a ±:L , i ^ j} 

9: else Define a ext = a 

^ i} 

where ir m such that i5 m = max{<5i j£ | i = 1, . . . , Ni} 
10: end if 

11: for all w € a ex t do 

12: Compute S w = |c| — \c w \ 

13: end for 

14: if c" 1 = 6 then output inverse of iro and stop 

15: if S w > then o length has been decreased 

16: Add (c w ,xw) to S 

17: end if 

18: end while 

19: Otherwise, output FAIL t> no more element to conjugate 



4.1.3. LBA with memory 1. Based on [5], we present another variant of LBA that 
is based on a fixed-size memory allocated for the algorithm. Here, S holds M 
tuples every round and is sorted by the first element (with respect to the length 
of conjugated element) of each tuple. In every round, the smallest element of S is 
removed and conjugated by all the generators and their inverses, the conjugated 
tuples are added back into S depending on whether there is still a free place in 
S. If there is no more places in S, and if the conjugated tuple is smaller than the 
largest element in S, swap them, and then re-sort S. Note that since S is always 
kept sorted, any operation to find the "smallest element" costs constant time. We 
use a time-out that can be defined as the halting condition. 

4.1.4. LBA with memory 2. A different algorithm, truer to the spirit of [5], is also 
considered. In this algorithm again, S holds M tuples every round. In every round, 
all elements of S are conjugated, but only the M smallest conjugated tuples (with 
respect to their length) are added back into S. Here again, for the halting condition, 
we use a time-out that is defined by the user (as in the previous variant). 

4.2. The length function. In the implementation of LBA, the choice of the length 
function is important (see [SJ[7]). In our case, the length of a word is chosen to be 
the sum of the absolute values of the exponents in its normal form. We choose this 
function because the experimental results presented below show that it satisfies the 
requirement £(a~ 1 ba) ^> 1(b) (as needed for a length function associated to LBA). 

The experiments are done by first constructing a polycyclic group G of Hirsch 
length h(G) following the construction in Section I57T1 below. Then, an element b 
of length between 10 and 13 is randomly chosen; we choose elements of this length 
for consistency in the length-based attack parameters. Another random element a 



6 DAVID GARBER, DELARAM KAHROBAEI, HA T. LAM 



Algorithm 3 LBA with Memory 1 

l: Initialize S = {(|F|,F,id G )}. 

2: while not time-out do 

3: Choose (|c|,c, x) £ S such that \c\ is minimal. Remove (|c|,c, x) 
4: for i = 1, . . . , Ni and e = ±1 do 
5: Compute c ai 

6: if c Qi = b then output inverse of xa\ and stop 

7: if Size(S) < M then 

8: Add (|c ai I , c ai , xaf) to S and sort 5 1 by first element of every tuple 

9: else > no more space in S 

10: if \c ai | is smaller than first element of all tuples in S then swap them 

11: end if 

12: end for 

13: end while 

14: Otherwise, output FAIL > no more element to conjugate 



Algorithm 4 LBA with Memory 2 

1: Initialize S = {(|F|,F,id G )}. 

2: while not time-out do 

3: for (\c\,c, x) G S do 

4: Remove (|c|,c, x) from S 

5: Compute c a * for all i G {1 . . . JVi} and e = ±1 

6: if c ai = 6 then output inverse of xaf and stop 

7: Save (|e°*| ,c a s:raf) in 5' 

8: end for 

9: After finished all conjugations, sort S' by the first element of every tuple 
10: Copy the smallest M elements into S and delete the rest of S' 

11: end while 

12: Otherwise, output FAIL 



of the same length interval is chosen and b a is computed, and finally, we compute 
|6 a | — We ran 100 tests for each group and the average difference is recorded. 



Polynomial 


h(G) 


Average difference 


x' 2 — x — 1 


3 


79.92 


x 5 — x 3 — 1 


7 


80.17 


x 11 - x 3 - 1 


16 


44.93 



As we can see, the average difference is large, in particular, |6 a | — 16| is significantly 
larger than |a|, indicating that the condition £(a~ 1 ba) 3> £(b) is indeed satisfied. 



5. Experimental Results 

Our goal is to test the feasibility of the LBA on AAG based on the polycyclic 
group. To that end, we implemented the four variants of the LBA presented in 
Section |4] and ran experiments on polycyclic groups with different Hirsch lengths. 
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5.1. Implementation details. Each polycyclic group is generated by choosing 
an irreducible polynomial / over Z[x], then / defines an algebraic field F over Q. 
Let Of be its maximal order and Uf its unit group, then Of x Uf is the desired 
polycyclic group. This construction follows 8 and is a part of the Polycyclic 
Package of GAP @]. 

A random element a$, for Alice's public set, or b{, for Bob's public set, is gen- 
erated by taking either some random generators of the group or their inverses and 
multiplying them together, while maintaining that the length of the element is be- 
tween a predefined minimum and maximum. By this method, we have more control 
over the length of the element. 

Alice's private key A is generated by taking a fixed number of random elements 
in a — (ai, . . . , djVi) and multiplying them together. Here we forgo control over 
length to preserve interesting cases of conjugations actually decreasing the length 
of hi, i.e. a commutator- type peak. 

5.2. Results. We performed several sets of tests, all of which were run on an Intel 
Core 17 quad-core 2.0GHz computer with 12GB of RAM, running Ubuntu version 
12.04 with GAP version 4.5 and 10GB of memory allowance. In all these tests, the 
polycyclic group G having Hirsch length h(G) is constructed by the above method 
with polynomial /. The size of Alice's and Bob's public sets are both Ni = N 2 = 20. 

5.2.1. The effect of the Hirsch length. In the first set of tests, the length of each 
random element o, or 6j is in the interval [L!,L 2 ] = [10, 13] and Alice's private 
key is the product of L — 5 random elements in Alice's public set. The time for 
each batch of 100 tests are recorded together with its success rate. In each case, a 
time-out of 30 minutes is enforced for each test. The following results are obtained 
by Algorithm [2j 



Polynomial 


h(G) 


Time 


Success rate 


x' 2 — x — 1 


3 


0.20 hours 


100% 


x 5 — x 3 — 1 


7 


76.87 hours 


35% 


x 7 — x 3 — 1 


10 


94.43 hours 


8% 


x 9 - 7x 3 - 1 


14 


95.18 hours 


5% 


x 11 — x 3 — 1 


16 


95.05 hours 


5% 



From this table, we can see that with a small Hirsch length, the length-based 
attack breaks A AG easily with high success rate. However, as the Hirsch length is 
increased to 7, the success rate decreases. The tipping point is Hirsch length 10, 
where the success rate is only 8%. At polycyclic groups with higher Hirsch lengths, 
we can see the effect of the time-out more prominently as the total time did not 
increase much more, but the success rate is dropped to only 5%. 

5.2.2. The effect of the key length. In the second set of tests, we vary the number of 
elements L that compose Alice's private key. Myasnikov and Ushakov [11 suggested 
that the LB A with a dynamic set (Algorithm ^ has a high success rate with long 
generators, i.e. random elements have longer length [L l5 L 2 ]. Therefore, we also 
vary the length of random elements according to the parameters in |14) . The 
following results are obtained by Algorithm [21 also with a time-out of 30 minutes. 
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Polynomial 


h(G) 


[10,13] 


[20,23] 


[40,43] 






L= 10 


L = 10 


L = 20 


L = 50 


x' — x A — 1 


10 


2% 


0% 


0% 


0% 


x 9 - 7x 3 - 1 


14 


0% 


0% 


0% 


0% 


x 11 - 3a; 3 - 1 


17 


0% 


0% 


0% 


0% 



The results of this set of tests indicate that just by increasing the number of 
generators of Alice's private key from 5 (as in the previous set of tests) to 10, the 
LBA already fails with polycyclic groups having Hirsch length as small as 10. 

5.2.3. Comparing the four variants of the LBA. In this paper, we compare the 
success rate of the four variants of the LBA for the first time on any platform. For 
comparing the success rate of the four variants of the LBA, we purposely choose the 
test parameters very small in this set of tests. They are as follows: N\ = N2 = 20, 
[L\, L2] = [5,8], L — 5, there is a time-out of 30 minutes and a memory of size 
M = 500. The polynomial used is f — x 3 — x — 1, generating a polycyclic group of 
Hirsch length 4. 



Algorithm 


Time 


Success rate 


Algorithm 1 (LBA with backtracking) 


0.57 hours 


58% 


Algorithm 2 (LBA with a dynamic set) 


37.35 hours 


95% 


Algorithm 3 (Memory 1) 


32.00 hours 


36% 


Algorithm 4 (Memory 2) 


4.01 hours 


92% 



Algorithm [2] gives the best success rate but took much longer than Algorithm [4] 
which gives a similar success rate in much shorter time. We conclude that with a 
sufficient size of memory, Algorithm |4] is the best variant of the LBA. 

5.2.4. Using the four variants of the LBA on our test parameters. In the fourth set 
of tests, we want to see the effect of the four different variants of the LBA presented 
in Section \A . 1 1 applied to our test parameters. Therefore, we keep the same following 
parameters for all the algorithms: the length of each random element is in the 
interval [L±, L2] = [10, 13], Alice's private key is the product of 10 elements and the 
length of both public sets are N± — N2 = 20. There is a time-out of 30 minutes 
per test and in the case of the two memory variants of the LBA, Algorithm [3] and 
Algorithm 2J a memory of size M = 1000 is used. The same polycyclic group G 
having Hirsch length 14 constructed from the polynomial x 9 — 7x 3 — 1 is used for 
all the variants of the LBA. 



Algorithm 


Time 


Success rate 


Algorithm Q] (LBA with backtracking) 


48.68 hours 


0% 


Algorithm [2] (LBA with a dynamic set) 


50.04 hours 


0% 


Algorithm [3] (Memory 1) 


50.00 hours 


0% 


Algorithm 0] (Memory 2) 


49.35 hours 


3% 



As we can see, Algorithm @] has the best performance in this set of parameters, 
but even then, it has only 3% success rate. To further test Algorithm 01 we ran 
another set of tests where we increase the length of random elements to [Lx, L2] — 
[20, 23] and increase the number of factors of the private key to L = 20. To give it 
a chance of success, we increase the size of the memory M to 40,000. The result is 
still 0% success rate. 
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5.2.5. The effect of increasing the time-out. Since it is possible that the time-out 
of 30 minutes for each test is not enough, we ran another set of tests where the 
time-out is 4 hours for each test. Algorithm 0] showed the most promise, so we chose 
it with the following parameters: the length of random elements is in the interval 
[Li, L2] = [20, 23], the number of factors of the private key is L = 20 and the size 
of the memory M is 1000. The polynomial used is x 9 — 7x 3 — 1 with Hirsch length 
14. Due to the long time-out, we performed only 50 tests. We still get 0% success 
rate. 

Based on the above experimental results, we conclude that polycyclic groups of 
high Hirsch lengths are resistant to the length-based attack. 

5.2.6. Additional experimental results concerning Algorithm^ These are some ad- 
ditional experimental results conducted with a time-out of 1 hour for each test. The 
polynomials used are / and h{G) is the Hirsch length of the generated polycyclic 
group. The size of Alice's and Bob's public sets are N\,N% respectively. Each ran- 
dom element ai or bi has length in [Li,L,2\ and Alice's private key is the product 
of L = 5 random elements in Alice's public set. The success rate of a batch of 100 
tests is recorded. 



Polynomial 


h(G) 


N, = 


N 2 = 5 


N 1 = N 2 = 20 






[5,8] 


[15,18] 


[10,13] 


x - 1 


1 


98% 




98% 


x 2 — x — 1 


3 


98% 


96% 


100% 


x 3 — x — 1 


4 


95% 




100% 


x 5 - x 3 - 1 


7 






35% 


x 7 — x 3 — 1 


10 






8% 


x 9 - 7x 3 - 1 


14 






5% 


x 11 - x 3 - 1 


16 


59% 


53% 


5% 
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